Length of Stay Prediction Models for Oral Cancer Surgery: Machine Learning, Statistical and ACS-NSQIP.

OBJECTIVE: Accurate prediction of hospital length of stay (LOS) following surgical management of oral cavity cancer (OCC) may be associated with improved patient counseling, hospital resource utilization and cost. The objective of this study was to compare the performance of statistical models, a machine learning (ML) model, and The American College of Surgeons National Surgical Quality Improvement Program's (ACS-NSQIP) calculator in predicting LOS following surgery for OCC. MATERIALS AND METHODS: A retrospective multicenter database study was performed at two major academic head and neck cancer centers. Patients with OCC who underwent major free flap reconstructive surgery between January 2008 and June 2019 surgery were selected. Data were pooled and split into training and validation datasets. Statistical and ML models were developed, and performance was evaluated by comparing predicted and actual LOS using correlation coefficient values and percent accuracy. RESULTS: Totally 837 patients were selected with mean patient age being 62.5 ± 11.7 [SD] years and 67% being male. The ML model demonstrated the best accuracy (validation correlation 0.48, 4-day accuracy 70%), compared with the statistical models: multivariate analysis (0.45, 67%) and least absolute shrinkage and selection operator (0.42, 70%). All were superior to the ACS-NSQIP calculator's performance (0.23, 59%). CONCLUSION: We developed statistical and ML models that predicted LOS following major free flap reconstructive surgery for OCC. Our models demonstrated superior predictive performance to the ACS-NSQIP calculator. The ML model identified several novel predictors of LOS. These models must be validated in other institutions before being used in clinical practice. LEVEL OF EVIDENCE: Level 3 Laryngoscope, 2024.

Machine learning in the prediction of massive transfusion in trauma: a retrospective analysis as a proof-of-concept.

PURPOSE: Early administration and protocolization of massive hemorrhage protocols (MHP) has been associated with decreases in mortality, multiorgan system failure, and number of blood products used. Various prediction tools have been developed for the initiation of MHP, but no single tool has demonstrated strong prediction with early clinical data. We sought to develop a massive transfusion prediction model using machine learning and early clinical data. METHODS: Using the National Trauma Data Bank from 2013 to 2018, we included severely injured trauma patients and extracted clinical features available from the pre-hospital and emergency department. We subsequently balanced our dataset and used the Boruta algorithm to determine feature selection. Massive transfusion was defined as five units at 4 h and ten units at 24 h. Six machine learning models were trained on the balanced dataset and tested on the original. RESULTS: A total of 326,758 patients met our inclusion with 18,871 (5.8%) requiring massive transfusion. Emergency department models demonstrated strong performance characteristics with mean areas under the receiver-operating characteristic curve of 0.83. Extreme gradient boost modeling slightly outperformed and demonstrated adequate predictive performance with pre-hospital data only, as well as 4-h transfusion thresholds. CONCLUSIONS: We demonstrate the use of machine learning in developing an accurate prediction model for massive transfusion in trauma patients using early clinical data. This research demonstrates the potential utility of artificial intelligence as a clinical decision support tool.

It's not the arrow, it's the archer: the role of the surgeon leader in a safety driven-era.

BACKGROUND: In an era where team communication and patient safety are paramount, standardized tools have been deemed critical to safe, efficient practice. In some cases-perhaps most notably in the surgical safety checklist (SSC)-these tools have been elevated as the key to safe patient care. However, effects of the SSC on patient safety in practice remain mixed. We explore the role and impact of the surgeon leader in the use of structured communication tools to understand how surgeon engagement impacts intraoperative teamwork. METHODS: Using a constructivist grounded theory approach, OR staff members (surgeons, anesthetists, nurses and perfusionists) were recruited to participate in a one-on-one semi-structured interview. The interview explored participant experiences working in the OR, focusing on the role and impact of the surgeon as leader. RESULTS: Engaged use of the surgical safety checklist by the attending surgeon had the potential to improve teamwork in the operating room. Surgeons who used the checklist to engage with their team and facilitate group discussion were able to avoid tensions later in the operation typically arising from lack of situation awareness and familiarity with team member experience levels. Surgeons who engaged with the SSC as more than a memory aid were able to foster a better team environment. CONCLUSIONS: Surgeons can harness their role as leader in the operating room by engaging with structured communication tools such as the SSC to foster improved teamwork.

Improving estimates of pertussis burden in Ontario, Canada 2010-2017 by combining validation and capture-recapture methodologies.

An underestimation of pertussis burden has impeded understanding of transmission and disallows effective policy and prevention to be prioritized and enacted. Capture-recapture analyses can improve burden estimates; however, uncertainty remains around incorporating health administrative data due to accuracy limitations. The aim of this study is to explore the impact of pertussis case definitions and data accuracy on capture-recapture estimates. We used a dataset from March 7, 2010 to December 31, 2017 comprised of pertussis case report, laboratory, and health administrative data. We compared Chao capture-recapture abundance estimates using prevalence, incidence, and adjusted false positive case definitions. The latter was developed by removing the proportion of false positive physician billing code-only case episodes after validation. We calculated sensitivity by dividing the number of observed cases by abundance. Abundance estimates demonstrated that a high proportion of cases were missed by all sources. Under the primary analysis, the highest sensitivity of 78.5% (95% CI 76.2-80.9%) for those less than one year of age was obtained using all sources after adjusting for false positives, which dropped to 43.1% (95% CI 42.4-43.8%) for those one year of age or older. Most code-only episodes were false positives (91.0%), leading to considerably lower abundance estimates and improvements in laboratory testing and case report sensitivity using this definition. Accuracy limitations can be accounted for in capture-recapture analyses using different case definitions and adjustment. The latter enhanced the validity of estimates, furthering the utility of capture-recapture methods to epidemiological research. Findings demonstrated that all sources consistently fail to detect pertussis cases. This is differential by age, suggesting ascertainment and testing bias. Results demonstrate the value of incorporating real time health administrative data into public health surveillance if accuracy limitations can be addressed.

Deep Learning Model for Automated Trainee Assessment During High-Fidelity Simulation.

PROBLEM: Implementation of competency-based medical education has necessitated more frequent trainee assessments. Use of simulation as an assessment tool is limited by access to trained examiners, cost, and concerns with interrater reliability. Developing an automated tool for pass/fail assessment of trainees in simulation could improve accessibility and quality assurance of assessments. This study aimed to develop an automated assessment model using deep learning techniques to assess performance of anesthesiology trainees in a simulated critical event. APPROACH: The authors retrospectively analyzed anaphylaxis simulation videos to train and validate a deep learning model. They used an anaphylactic shock simulation video database from an established simulation curriculum, integrating a convenience sample of 52 usable videos. The core part of the model, developed between July 2019 and July 2020, is a bidirectional transformer encoder. OUTCOMES: The main outcome was the F1 score, accuracy, recall, and precision of the automated assessment model in analyzing pass/fail of trainees in simulation videos. Five models were developed and evaluated. The strongest model was model 1 with an accuracy of 71% and an F1 score of 0.68. NEXT STEPS: The authors demonstrated the feasibility of developing a deep learning model from a simulation database that can be used for automated assessment of medical trainees in a simulated anaphylaxis scenario. The important next steps are to (1) integrate a larger simulation dataset to improve the accuracy of the model; (2) assess the accuracy of the model on alternative anaphylaxis simulations, additional medical disciplines, and alternative medical education evaluation modalities; and (3) gather feedback from education leadership and clinician educators surrounding the perceived strengths and weaknesses of deep learning models for simulation assessment. Overall, this novel approach for performance prediction has broad implications in medical education and assessment.

Exploring the Use of Natural Language Processing for Objective Assessment of Disorganized Speech in Schizophrenia.

Objective: Measurement-based care tools in psychiatry are useful for symptom monitoring and detecting response to treatment, but methods for quick and objective measurement are lacking especially for acute psychosis. The aim of this study was to explore potential language markers, detected by natural language processing (NLP) methods, as a means to objectively measure the severity of psychotic symptoms of schizophrenia in an acute clinical setting. Methods: Twenty-two speech samples were collected from seven participants who were hospitalized for schizophrenia, and their symptoms were evaluated over time with SAPS/SANS and TLC scales. Linguistic features were extracted from the speech data using machine learning techniques. Spearman's correlation was performed to examine the relationship between linguistic features and symptoms. Various machine learning models were evaluated by cross-validation methods for their ability to predict symptom severity using the linguistic markers. Results: Reduced lexical richness and syntactic complexity were characteristic of negative symptoms, while lower content density and more repetitions in speech were predictors of positive symptoms. Machine learning models predicted severity of alogia, illogicality, poverty of speech, social inattentiveness, and TLC scores with up to 82% accuracy. Additionally, speech incoherence was quantifiable through language markers derived from NLP methods. Conclusions: These preliminary findings suggest that NLP may be useful in identifying clinically relevant language markers of schizophrenia, which can enhance objectivity in symptom monitoring during hospitalization. Further work is needed to replicate these findings in a larger data set and explore methods for feasible implementation in practice.

Data Science as a Core Competency in Undergraduate Medical Education in the Age of Artificial Intelligence in Health Care.

The increasingly sophisticated and rapidly evolving application of artificial intelligence in medicine is transforming how health care is delivered, highlighting a need for current and future physicians to develop basic competency in the data science that underlies this topic. Medical educators must consider how to incorporate central concepts in data science into their core curricula to train physicians of the future. Similar to how the advent of diagnostic imaging required the physician to understand, interpret, and explain the relevant results to patients, physicians of the future should be able to explain to patients the benefits and limitations of management plans guided by artificial intelligence. We outline major content domains and associated learning outcomes in data science applicable to medical student curricula, suggest ways to incorporate these themes into existing curricula, and note potential implementation barriers and solutions to optimize the integration of this content.

Brain imaging signatures of neuropathic facial pain derived by artificial intelligence.

Advances in neuroimaging have permitted the non-invasive examination of the human brain in pain. However, a persisting challenge is in the objective differentiation of neuropathic facial pain subtypes, as diagnosis is based on patients' symptom descriptions. We use artificial intelligence (AI) models with neuroimaging data to distinguish subtypes of neuropathic facial pain and differentiate them from healthy controls. We conducted a retrospective analysis of diffusion tensor and T1-weighted imaging data using random forest and logistic regression AI models on 371 adults with trigeminal pain (265 classical trigeminal neuralgia (CTN), 106 trigeminal neuropathic pain (TNP)) and 108 healthy controls (HC). These models distinguished CTN from HC with up to 95% accuracy, and TNP from HC with up to 91% accuracy. Both classifiers identified gray and white matter-based predictive metrics (gray matter thickness, surface area, and volume; white matter diffusivity metrics) that significantly differed across groups. Classification of TNP and CTN did not show significant accuracy (51%) but highlighted two structures that differed between pain groups-the insula and orbitofrontal cortex. Our work demonstrates that AI models with brain imaging data alone can differentiate neuropathic facial pain subtypes from healthy data and identify regional structural indicates of pain.

Contextualizing the tone of the operating room in practice: drawing on the literature to connect the dots.

The study of teamwork in the operating room has made significant strides in uncovering key constructs which shape safe and effective intraoperative care. However, in recent years, there have been calls to understand teamwork in the operating room more fully by embracing the complexity of the intraoperative environment. We propose the construct of tone as a useful lens through which to understand intraoperative teamwork. In this article, we review the literature on culture, shared mental models, and psychological safety, linking each to the construct of tone. By identifying tone as a theoretical orientation to demonstrate the overlap between these concepts, we aim to provide a starting point for new ways to understand intraoperative team dynamics.

Developing a reference standard for pertussis by applying a stratified sampling strategy to electronic medical record data.

PURPOSE: Pertussis surveillance remains essential in Canada, but ascertainment bias limits the accuracy of surveillance data. Introducing other sources to improve detection has highlighted the importance of validation. However, challenges arise due to low prevalence, and oversampling suspected cases can introduce partial verification bias. The aim of this study was to build a reference standard for pertussis validation studies that provides adequate analytic precision and minimizes bias. METHODS: We used a stratified strategy to sample the reference standard from a primary care electronic medical record cohort. We incorporated abstractor notes into definite, possible, ruled-out, and no mention of pertussis classifications which were based on surveillance case definitions. RESULTS: We abstracted eight hundred records from the cohort of 404,922. There were 208 (26%) definite and 261 (32.6%) possible prevalent pertussis cases. Classifications demonstrated a wide variety of case severities. Abstraction reliability was moderate to substantial based on Cohen's kappa and raw percent agreement. CONCLUSIONS: When conducting validation studies for pertussis and other low prevalence diseases, this stratified sampling strategy can be used to develop a reference standard using limited resources. This approach mitigates verification and spectrum bias while providing sufficient precision and incorporating a range of case severities.

Validating pertussis data measures using electronic medical record data in Ontario, Canada 1986-2016.

Background: Pertussis is a reportable disease in many countries, but ascertainment bias has limited data accuracy. This study aims to validate pertussis data measures using a reference standard that incorporates different suspected case severities, allowing for the impact of case severity on accuracy and detection to be explored. Methods: We evaluated 25 pertussis detection algorithms in a primary care electronic medical record database between January 1, 1986 and December 30, 2016. We estimated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We used sensitivity analyses to explore areas of uncertainty and evaluated reasons for lack of detection. Results: The algorithm including all data measures achieved the highest sensitivity at 20.6%. Sensitivity increased to 100% after reclassifying symptom-only cases as non-cases, but the PPV remained low. Age at first episode was significantly associated with detection in half of the tested scenarios, and false negatives often had some history of immunization. Conclusions: Sensitivity improved by reclassifying symptom-only cases but remained low unless multiple data sources were used. Results demonstrate a trade-off between PPV and sensitivity. EMRs can enhance detection through patient history and clinical note data. It is essential to improve case identification of older individuals with vaccination history to reduce ascertainment bias.

Conformal mirror descent with logarithmic divergences.

The logarithmic divergence is an extension of the Bregman divergence motivated by optimal transport and a generalized convex duality, and satisfies many remarkable properties. Using the geometry induced by the logarithmic divergence, we introduce a generalization of continuous time mirror descent that we term the conformal mirror descent. We derive its dynamics under a generalized mirror map, and show that it is a time change of a corresponding Hessian gradient flow. We also prove convergence results in continuous time. We apply the conformal mirror descent to online estimation of a generalized exponential family, and construct a family of gradient flows on the unit simplex via the Dirichlet optimal transport problem.

A Delphi consensus statement for digital surgery.

The use of digital technology is increasing rapidly across surgical specialities, yet there is no consensus for the term 'digital surgery'. This is critical as digital health technologies present technical, governance, and legal challenges which are unique to the surgeon and surgical patient. We aim to define the term digital surgery and the ethical issues surrounding its clinical application, and to identify barriers and research goals for future practice. 38 international experts, across the fields of surgery, AI, industry, law, ethics and policy, participated in a four-round Delphi exercise. Issues were generated by an expert panel and public panel through a scoping questionnaire around key themes identified from the literature and voted upon in two subsequent questionnaire rounds. Consensus was defined if >70% of the panel deemed the statement important and <30% unimportant. A final online meeting was held to discuss consensus statements. The definition of digital surgery as the use of technology for the enhancement of preoperative planning, surgical performance, therapeutic support, or training, to improve outcomes and reduce harm achieved 100% consensus agreement. We highlight key ethical issues concerning data, privacy, confidentiality and public trust, consent, law, litigation and liability, and commercial partnerships within digital surgery and identify barriers and research goals for future practice. Developers and users of digital surgery must not only have an awareness of the ethical issues surrounding digital applications in healthcare, but also the ethical considerations unique to digital surgery. Future research into these issues must involve all digital surgery stakeholders including patients.

Are smartphones and machine learning enough to diagnose tremor?

BACKGROUND: Patients with essential tremor (ET), Parkinson's disease (PD) and dystonic tremor (DT) can be difficult to classify and often share similar characteristics. OBJECTIVES: To use ubiquitous smartphone accelerometers with and without clinical features to automate tremor classification using supervised machine learning, and to use unsupervised learning to evaluate if natural clusterings of patients correspond to assigned clinical diagnoses. METHODS: A supervised machine learning classifier was trained to classify 78 tremor patients using leave-one-out cross-validation to estimate performance on unseen accelerometer data. An independent cohort of 27 patients were also studied. Next, we focused on a subset of 48 patients with both smartphone-based tremor measurements and detailed clinical assessment metrics and compared two separate machine learning classifiers trained on these data. RESULTS: The classifier yielded a total accuracy of 74.4% and F1-score of 0.74 for a trinary classification with an area under the curve of 0.904, average F1-score of 0.94, specificity of 97% and sensitivity of 84% in classifying PD from ET or DT. The algorithm classified ET from non-ET with 88% accuracy, but only classified DT from non-DT with 29% accuracy. A poorer performance was found in the independent cohort. Classifiers trained on accelerometer and clinical data respectively obtained similar results. CONCLUSIONS: Machine learning classifiers achieved a high accuracy of PD, however moderate accuracy of ET, and poor accuracy of DT classification. This underscores the difficulty of using AI to classify some tremors due to lack of specificity in clinical and neuropathological features, reinforcing that they may represent overlapping syndromes.

Predicting the target specialty of referral notes to estimate per-specialty wait times with machine learning.

BACKGROUND: Currently, in Canada, existing health administrative data and hospital-inputted portal systems are used to measure the wait times to receiving a procedure or therapy after a specialist visit. However, due to missing and inconsistent labelling, estimating the wait time prior to seeing a specialist physician requires costly manual coding to label primary care referral notes. METHODS: In this work, we represent the notes using word-count vectors and develop a logistic regression machine learning model to automatically label the target specialist physician from a primary care referral note. These labels are not available in the administrative system. We also study the effects of note length (measured in number of tokens) and dataset size (measured in number of notes per target specialty) on model performance to help other researchers determine if such an approach may be feasible for them. We then calculate the wait time by linking the specialist type from a primary care referral to a full consultation visit held in Ontario, Canada health administrative data. RESULTS: For many target specialties, we can reliably (F1Score ≥ 0.70) predict the target specialist type. Doing so enables the automated measurement of wait time from family physician referral to specialist physician visit. Of the six specialties with wait times estimated using both 2008 and 2015 data, two had a substantial increase (defined as a change such that the original value lay outside the 95% confidence interval) in both median and 75th percentile wait times, one had a substantial decrease in both median and 75th percentile wait times, and three has non-substantial increases. CONCLUSIONS: Automating these wait time measurements, which had previously been too time consuming and costly to evaluate at a population level, can be useful for health policy researchers studying the effects of policy decisions on patient access to care.

Differential expression of a brain aging biomarker across discrete chronic pain disorders.

ABSTRACT: Chronic pain has widespread, detrimental effects on the human nervous system and its prevalence and burden increase with age. Machine learning techniques have been applied on brain images to produce statistical models of brain aging. Specifically, the Gaussian process regression is particularly effective at predicting chronological age from neuroimaging data which permits the calculation of a brain age gap estimate (brain-AGE). Pathological biological processes such as chronic pain can influence brain-AGE. Because chronic pain disorders can differ in etiology, severity, pain frequency, and sex-linked prevalence, we hypothesize that the expression of brain-AGE may be pain specific and differ between discrete chronic pain disorders. We built a machine learning model using T1-weighted anatomical MRI from 812 healthy controls to extract brain-AGE for 45 trigeminal neuralgia (TN), 52 osteoarthritis (OA), and 50 chronic low back pain (BP) subjects. False discovery rate corrected Welch t tests were conducted to detect significant differences in brain-AGE between each discrete pain cohort and age-matched and sex-matched controls. Trigeminal neuralgia and OA, but not BP subjects, have significantly larger brain-AGE. Across all 3 pain groups, we observed female-driven elevation in brain-AGE. Furthermore, in TN, a significantly larger brain-AGE is associated with response to Gamma Knife radiosurgery for TN pain and is inversely correlated with the age at diagnosis. As brain-AGE expression differs across distinct pain disorders with a pronounced sex effect for female subjects. Younger women with TN may therefore represent a vulnerable subpopulation requiring expedited chronic pain intervention. To this end, brain-AGE holds promise as an effective biomarker of pain treatment response.

Population-based incidence of invasive pneumococcal disease in children and adults in Ontario and British Columbia, 2002-2018: A Canadian Immunization Research Network (CIRN) study.

BACKGROUND: Invasive pneumococcal disease (IPD) burden, evaluated in Canada using reported confirmed cases in surveillance systems, is likely underestimated due to underreporting. We estimated the burden of IPD in Ontario and British Columbia (BC) by combining surveillance data with health administrative databases. METHODS: We established a cohort of 27,525 individuals in Ontario and BC. Laboratory-confirmed IPD cases were identified from Ontario's integrated Public Health Information System and the BC Centre for Disease Control Public Health Laboratory. Possible IPD cases were identified from hospitalization data in both provinces, and from emergency department visit data in Ontario. We estimated the age and sex adjusted annual incidence of IPD and pneumococcal conjugate/polysaccharide vaccine (PCV/PPV) serotype-specific IPD using Poisson regression models. RESULTS: In Ontario, 20,205 overall IPD cases, including 15,299 laboratory-confirmed cases, were identified with relatively stable age- and sex-adjusted annual incidence rates ranging from 13.7/100,000 (2005) to 13.6/100,000 (2018). In BC, 7,320 overall IPD cases, including 5,932 laboratory-confirmed cases were identified; annual incidence rates increased from 10.9/100,000 (2002) to 13.2/100,000 (2018). Older adults aged ≥ 85 years had the highest incidence rates. During 2007-2018 the incidence of PCV7 serotypes and additional PCV13 serotypes decreased while the incidence of unique PPV23 and non-vaccine serotypes increased in both provinces. CONCLUSIONS: IPD continues to cause a substantial public health burden in Canada despite publicly funded pneumococcal vaccination programs, resulting in part from an increase in unique PPV23 and non-vaccine serotypes.

BENDR: Using Transformers and a Contrastive Self-Supervised Learning Task to Learn From Massive Amounts of EEG Data.

Deep neural networks (DNNs) used for brain-computer interface (BCI) classification are commonly expected to learn general features when trained across a variety of contexts, such that these features could be fine-tuned to specific contexts. While some success is found in such an approach, we suggest that this interpretation is limited and an alternative would better leverage the newly (publicly) available massive electroencephalography (EEG) datasets. We consider how to adapt techniques and architectures used for language modeling (LM) that appear capable of ingesting awesome amounts of data toward the development of encephalography modeling with DNNs in the same vein. We specifically adapt an approach effectively used for automatic speech recognition, which similarly (to LMs) uses a self-supervised training objective to learn compressed representations of raw data signals. After adaptation to EEG, we find that a single pre-trained model is capable of modeling completely novel raw EEG sequences recorded with differing hardware, and different subjects performing different tasks. Furthermore, both the internal representations of this model and the entire architecture can be fine-tuned to a variety of downstream BCI and EEG classification tasks, outperforming prior work in more task-specific (sleep stage classification) self-supervision.

Regional brain morphology predicts pain relief in trigeminal neuralgia.

BACKGROUND: Trigeminal neuralgia, a severe chronic neuropathic pain disorder, is widely believed to be amenable to surgical treatments. Nearly 20% of patients, however, do not have adequate pain relief after surgery. Objective tools for personalized pre-treatment prognostication of pain relief following surgical interventions can minimize unnecessary surgeries and thus are of substantial benefit for patients and clinicians. PURPOSE: To determine if pre-treatment regional brain morphology-based machine learning models can prognosticate 1 year response to Gamma Knife radiosurgery for trigeminal neuralgia. METHODS: We used a data-driven approach that combined retrospective structural neuroimaging data and support vector machine-based machine learning to produce robust multivariate prediction models of pain relief following Gamma Knife radiosurgery for trigeminal neuralgia. Surgical response was defined as ≥ 75% pain relief 1 year post-treatment. We created two prediction models using pre-treatment regional brain gray matter morphology (cortical thickness or surface area) to distinguish responders from non-responders to radiosurgery. Feature selection was performed through sequential backwards selection algorithm. Model out-of-sample generalizability was estimated via stratified 10-fold cross-validation procedure and permutation testing. RESULTS: In 51 trigeminal neuralgia patients (35 responders, 16 non-responders), machine learning models based on pre-treatment regional brain gray matter morphology (14 regional surface areas or 13 regional cortical thicknesses) provided robust a priori prediction of surgical response. Cross-validation revealed the regional surface area model was 96.7% accurate, 100.0% sensitive, and 89.1% specific while the regional cortical thickness model was 90.5% accurate, 93.5% sensitive, and 83.7% specific. Permutation testing revealed that both models performed beyond pure chance (p < 0.001). The best predictor for regional surface area model and regional cortical thickness model was contralateral superior frontal gyrus and contralateral isthmus cingulate gyrus, respectively. CONCLUSIONS: Our findings support the use of machine learning techniques in subsequent investigations of chronic neuropathic pain. Furthermore, our multivariate framework provides foundation for future development of generalizable, artificial intelligence-driven tools for chronic neuropathic pain treatments.

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech.

Introduction: Research related to the automatic detection of Alzheimer's disease (AD) is important, given the high prevalence of AD and the high cost of traditional diagnostic methods. Since AD significantly affects the content and acoustics of spontaneous speech, natural language processing, and machine learning provide promising techniques for reliably detecting AD. There has been a recent proliferation of classification models for AD, but these vary in the datasets used, model types and training and testing paradigms. In this study, we compare and contrast the performance of two common approaches for automatic AD detection from speech on the same, well-matched dataset, to determine the advantages of using domain knowledge vs. pre-trained transfer models. Methods: Audio recordings and corresponding manually-transcribed speech transcripts of a picture description task administered to 156 demographically matched older adults, 78 with Alzheimer's Disease (AD) and 78 cognitively intact (healthy) were classified using machine learning and natural language processing as "AD" or "non-AD." The audio was acoustically-enhanced, and post-processed to improve quality of the speech recording as well control for variation caused by recording conditions. Two approaches were used for classification of these speech samples: (1) using domain knowledge: extracting an extensive set of clinically relevant linguistic and acoustic features derived from speech and transcripts based on prior literature, and (2) using transfer-learning and leveraging large pre-trained machine learning models: using transcript-representations that are automatically derived from state-of-the-art pre-trained language models, by fine-tuning Bidirectional Encoder Representations from Transformer (BERT)-based sequence classification models. Results: We compared the utility of speech transcript representations obtained from recent natural language processing models (i.e., BERT) to more clinically-interpretable language feature-based methods. Both the feature-based approaches and fine-tuned BERT models significantly outperformed the baseline linguistic model using a small set of linguistic features, demonstrating the importance of extensive linguistic information for detecting cognitive impairments relating to AD. We observed that fine-tuned BERT models numerically outperformed feature-based approaches on the AD detection task, but the difference was not statistically significant. Our main contribution is the observation that when tested on the same, demographically balanced dataset and tested on independent, unseen data, both domain knowledge and pretrained linguistic models have good predictive performance for detecting AD based on speech. It is notable that linguistic information alone is capable of achieving comparable, and even numerically better, performance than models including both acoustic and linguistic features here. We also try to shed light on the inner workings of the more black-box natural language processing model by performing an interpretability analysis, and find that attention weights reveal interesting patterns such as higher attribution to more important information content units in the picture description task, as well as pauses and filler words. Conclusion: This approach supports the value of well-performing machine learning and linguistically-focussed processing techniques to detect AD from speech and highlights the need to compare model performance on carefully balanced datasets, using consistent same training parameters and independent test datasets in order to determine the best performing predictive model.